Focusing on the unbalance issue between local optimization and global optimization and the inability to jump out of the local optimum of Artificial Fish Swarm Algorithm (AFSA), an Adaptive AFSA utilizing Gene Exchange (AAFSA-GE) was proposed. Firstly, an adaptive mechanism of view and step was utilized to enhance the search speed and accuracy. Then, chaotic behavior and gene exchange behavior were employed to improve the ability of jumping out of the local optimum and the search efficiency. Ten classic test functions were selected to prove the feasibility and robustness of the proposed algorithm by comparing it with the other three modified AFSAs, which are Normative Fish Swarm Algorithm (NFSA), FSA optimized by PSO algorithm with Extended Memory (PSOEM-FSA), and Comprehensive Improvement of Artificial Fish Swarm Algorithm (CIAFSA). Experimental results show that AAFSA-GE achieves better results in local and global search ability than those of PSOEM-FSA and CIAFSA,and better search efficiency and better global search ability than those of NSFA.